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Developmental Assessment Scales (MIDAS)" (Albert Wiswell et al.) explores 
MIDAS's psychometric saliency. Findings indicates this instrument represents 
an incomplete attempt to develop a valid assessment of multiple 
intelligences; further work conceptually refining some subscales may lead to 
more robust measures of these constructs. "Evaluating Corporate Quality of 
Life (QOL) Programs: Utilizing the Balanced Scorecard Model in Conjunction 
with Quasi -Experimental Design" (Wendy J. Casper, Ann L. Landy) describes the 
development of a performance monitoring system for the QOL programs of a 
large, decentralized government agency, that was designed using the Balanced 
Scorecard model for strategic planning, measurement, and program management 
in conjunction with a synthesis of research findings. "Validating a Tool to 
Measure Customer Service Competencies" (Darlene Russ-Eft, Caryl Berrey, Ruth 
Pangilinan, Regina Atwood) reports a study that used the critical incidents 
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develop and validate a tool to assess service competencies. All three papers 
include bibliographies. (YLB) 
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Wider acceptance and use of Multiple Intelligence theory by psychologists, educators, and 
trainers have been thwarted by the lack of a practical, reliable and valid method of assessment. 

When used as a diagnostic and combined with behavioral observations, the MIDAS instrument 
could be useful in facilitating career choices and tailoring learning opportunities to multiple 
means of understanding. The purpose of this study was to explore the MIDAS’s psychometric 
saliency. 

Key Words: Multiple Intelligence, Measurement, MIDAS 

What relevance is intelligence to education and training? Would it be helpful to diagnose, prescreen, and or predict 
some measure of success in specific fields of endeavor? This paper assumes its obvious relevance and contribution. 
Our attention herein is upon the measurement of intelligence that has become synonymous with the construct in our 
western culture. Measurement (of intelligence) has been a major focus since Alfred Binet developed the first test of 
cognitive abilities for French school children in 1905. In spite of Binet's refusal to claim that his test actually 
measured intelligence, his invention became the basis for one of psychology’s most enduring contributions to 
education and society of the 20th century (Binet & Simon, 1916). In the wake of Binet’s work, the unitary 
definition of intelligence — as exemplified by the intelligence quotient (I.Q.) and assessed by the Stanford-Binet, 
Weschler and college admission tests — has become a commonplace. 

Although prominent theorists such as J.P. Guilford (1967), L.L. Thurstone (1947) and R.J. Sternberg 
(1988) have offered multi-factored alternatives to the unitary I.Q. model, none have had the impact on the 
educational community as that of the theory of multiple intelligences (MI). 

In 1983, Howard Gardner proposed that human intelligence is best defined as a "bio-social 
potential" that allows a person to solve problems and create products that are valued within a community. Using this 
definition and eight criteria Gardner has identified at least eight distinct yet complementary constructs: linguistic, 
logical-mathematical, musical, spatial, kinesthetic, interpersonal and intrapersonal and naturalist (1993). 

Figure 1. Gardner Constructs represented by the MIDAS 
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The Multiple Intelligences Developmental Assessment Scales (MIDAS) 

Wider acceptance and use of MI theory by psychologists, educators, and trainers has been thwarted by the lack of a 
practical, reliable and valid method of assessment. MI theory itself has likewise been viewed with suspicion by 
traditional cognitive scientists accustomed to assuming that intelligence can be quantified via a single I.Q. score. 
Critics of MI theory dismiss its legitimacy by claiming that it is more of a "literary" rather than "scientific" theory 
because there is no valid method of measurement that produces quantifiable data. Gardner's contextual and 
culturally embedded definition of intelligence challenges the assumption that intellectual prowess can be reduced to 
a single score that can be measured via paper-and-pencil and decontextualized tests. In the MI perspective, the vast 
majority of "real world" problem-solving occurs within value-laden, stimulus-rich situations requiring something 
more than answering short questions with short answers as quickly as possible. This contextually-based conception 
of intellectual ability requires a contextual approach to assessment in order to describe the person's intellectual and 
creative life (Shearer, 1994). 

The Multiple Intelligences Developmental Assessment Scales (MIDAS) were originally created in 1986 to 
be administered to adults in conjunction with a neuropsychological evaluation. It was first developed as an interview 
to be conducted with a family member to. obtain an understanding of the intellectual profile of the patient (Shearer, 
1994). Uses since its conversion to a paper pencil questionnaire are now targeted at adult populations to quickly self- 
assess one’s MI profile for counseling and educational uses. Because of the potential for the instrument to gain 
popularity and utility with educators, trainers, and HR practitioners, an objective assessment of the MIDAS’s 
reliability and internal psychometric constructs (as initial evidence what it purports to measure is as important as its 
theoretical framework and potential applications. 

According to Shearer (1994), its conversion to a 119 item self-completed questionnaire was designed to 
retain features of a structured interview probing for both qualitative information and quantitative data. On the 
questionnaire itself, response choices are marked by letter rather than by number to encourage respondents to 
respond to the descriptive choice rather than to a number. Likewise, the descriptors for each response option are 
uniquely written to fit the specific content for each question. 

The questions inquire about observable activities that represent core features of each of the multiple 
intelligences as described by Howard Gardner. The questions are written in one of three basic forms. A majority of 
questions ask the informant to provide a realistic evaluation of his/her skill level in a specified activity. The second 
type of questions inquire about the frequency or duration of time the respondent spends engaged in an activity 
representative of the designated construct. The fewest number of questions ask for an assessment of the respondent’s 
enthusiasm for the target activity. The MIDAS Profile consists of percentage scores for the eight areas of 
intelligence along with 29 sub-scale scores describing specific skills. Scale scores are calculated and used to profile 
respondents. 

The wording of questions; scale composition and scoring matrices went through a lengthy series revisions 
and testing from 1987 through 1996 (Shearer, 1994). An initial series of small scale studies reported acceptable 
psychometric properties including factor structure, item consistency, test-retest reliability, inter-rater agreement; 
subscale factor analysis and appropriate discrimination with various criterion groups and measures (Shearer, 1994; 
Buros, 1999). 

Purpose 

The MIDAS instrument was intended to help in providing a means to indicate relative interest in and one’s self - 
evaluation of abilities in regard to Gardner’s eight facets of intelligences. Though arguably not an achievement or 
aptitude scale, when used as a diagnostic and combined with behavioral observations, the MIDAS could be useful in 
facilitating career choices and tailoring learning opportunities to multiple means of understanding. The purpose of 
this correlational study is to help explore the MIDAS’s psychometric saliency by examining the reliability (internal 
consistency) and factor constructs of the instrument and its scales. Moreover, this study was to further examine the 
MIDAS scales using exploratory factor analytic techniques with a significantly larger sample than was available 
during the instrument’s development. 

Method 

Sample and Data Analyses. Sample size requirements are driven by recommended factor-analytic and structural 
equation modeling (SEM) conventions with regard to subject (orcase) to variable (or parameter) ratios and minimum 
size of samples. It is generally advocated that for factor-analytic work an acceptable or “rule of thumb” ratio should 
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be at least five cases per variable, with a minimum sample size to be 200 (Kline, 1993; Tabachnick & Fidell, 1989). 
Kline (1998) also recommends sample sizes of at least 200 for SEM analyses. To analyze the MIDAS’s 1 19 items, a 
total sample of 1419 adults were be used. A total of 1419 protocols were coded analyzed in StatView 5.0 (SAS) for 
internal reliability, principle component, and factor analyses. 

Psychometrics. To adequately examine the MIDAS, the analyses performed in this study followed very 
deliberate conventions (Nunnally & Bernstein, 1994; Tabachnick & Fidell, 1989): demographic frequency 
distribution analysis, an internal reliability analysis (Cronbach's alpha), an examination of the correlation matrix for 
factorability, the number of factors to retain were user specified based on MI theory, after which an iterative or 
common factor analysis was performed for the final extraction, factor stability comparisons, potential second order 
implications, and possible theoretical interpretations. 

Results 

Reliability. In this study, all eight scales yielded reliability coefficients of .85 to .90 (Cronbach Alpha) as depicted 
below in Table 1: 

Table 1. Reliability (MIDAS) 



Scales AlphaXoefficients 


Musical 


.87 


Physical 


.85 


Math/logic 


.85 


Spatial 


.87 


Linguistic 


.86 


Interpersonal 


.86 


Intrapersonal 


.90 


Naturalist 


.88 



Exploratory Factor Analysis. Based on the MIDAS’s theoretical framework (some expected commonality among 
constructs), eight factors were specified and a simple oblique solution was sought using an iterative or common 
factor extraction. With a varimax rotation, the following eigenvalues were produced: 15.2, 7.6, 5.1, 4.5, 3.8, 2.9, 
2.6, and 1 .7 respectively accounting for .36 of the variance. Factor loadings are depicted below in Table 2. 

Table 2. EFA, Eight Factors, Varimax Rotation, Oblique Solution 



Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 Factor 7 Factor 8 



Musical 


0.01 


0.15 


0.09 


0.58 


-0.17 


-0.08 


-0.09 


-0.02 


Musical 


0.10 


0.05 


0.04 


0.49 


-0.11 


-0.06 


-0.07 


0.01 


Musical 


0.03 


0.00 


-0.01 


0.53 


-0.08 


-0.04 


-0.09 


0.13 


Musical 


-0.03 


0.02 


0.05 


0.43 


-0.03 


-0.06 


-0.09 


0.14 


Musical 


0.09 


-0.04 


-0.04 


0.63 


-0.07 


-0.06 


-0.05 


0.07 


Musical 


0.01 


0.03 


-0.05 


0.61 


0.05 


0.01 


0.09 


-0.19 


Musical 


-0.02 


-0.13 


0.01 


0.55 


0.05 


0.10 


0.03 


0.04 


Musical 


-0.01 


-0.05 


0.00 


0.64 


0.13 


0.07 


0.02 


0.03 


Musical 


0.01 


-0.04 


-0.04 


0.68 


0.10 


0.08 


0.01 


0.00 


Musical 


0.00 


-0.08 


-0.06 


0.65 


0.05 


0.10 


0.03 


-0.08 


Musical 


0.04 


0.06 


-0.06 


0.51 


0.03 


-0.08 


0.01 


0.10 


Musical 


0.03 


0.01 


0.07 


0.53 


0.02 


0.05 


0.00 


-0.11 


Musical 


-0.03 


-0.05 


0.00 


0.56 


0.06 


0.08 


0.03 


0.00 


Musical 


0.01 


0.06 


-0.05 


0.53 


0.11 


0.01 


0.12 


-0.18 


Physical 


-0.05 


0.01 


-0.01 


-0.08 


0.78 


-0.11 


-0.08 


0.07 


Physical 


0.01 


-0.04 


0.04 


-0.04 


0.80 


0.01 


-0.08 


0.05 


Physical 


-0.06 


0.15 


-0.06 


0.29 


-0.08 


0.11 


0.01 


-0.02 
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Physical 


-0.02 


0.05 


0.02 


0.04 


0.66 


-0.02 


0.01 


0.04 


Physical 


-0.03 


0.17 


0.04 


0.09 


0.34 


0.00 


0.04 


-0.03 


Physical 


0.06 


-0.01 


-0.02 


-0.02 


0.81 


0.00 


-0.15 


0.03 


Physical 


0.05 


0.19 


0.15 


-0.03 


0.40 


-0.09 


0.05 


-0.12 


Physical 


0.11 


0.11 


0.00 


0.10 


0.31 


-0.17 


0.19 


0.13 


Physical 


0.11 


0.16 


0.00 


0.10 


-0.14 


-0.19 


0.48 


-0.04 


Physical 


0.05 


-0.04 


0.07 


-0.02 


0.08 


-0.08 


0.62 


0.03 


Physical 


-0.24 


0.08 


-0.03 


0.08 


0.22 


0.20 


0.31 


0.05 


Physical 


-0.14 


0.21 


-0.08 


0.20 


0.17 


-0.02 


0.15 


0.01 


Physical 


-0.05 


0.18 


-0.01 


0.06 


0.08 


-0.05 


0.14 


-0.01 


math/logic 


0.74 


0.01 


-0.08 


0.10 


-0.12 


-0.08 


-0.12 


-0.06 


math/logic 


0.80 


-0.03 


-0.03 


0.08 


-0.06 


-0.10 


-0.09 


-0.07 


math/logic 


0.67 


-0.01 


-0.04 


0.08 


-0.07 


-0.05 


-0.14 


-0.07 


math/logic 


0.45 


-0.22 


0.30 


-0.01 


0.05 


0.09 


0.08 


-0.02 


math/logic 


0.33 


0.02 


0.09 


0.05 


0.11 


-0.05 


0.10 


0.10 


math/logic 


0.48 


-0.06 


0.07 


0.00 


0.01 


0.05 


0.12 


0.09 


math/logic 


0.21 


0.06 


0.01 


0.15 


-0.13 


0.15 


0.04 


-0.05 


math/logic 


0.47 


0.05 


-0.05 


-0.05 


-0.05 


-0.07 


0.05 


0.02 


math/logic 


0.39 


0.07 


-0.13 


0.09 


-0.04 


0.03 


0.06 


0.11 


math/logic 


0.73 


-0.05 


-0.05 


0.04 


0.04 


0.06 


-0.08 


0.08 


math/logic 


0.44 


-0.06 


0.10 


0.02 


0.11 


0.07 


0.22 


0.05 


math/logic 


0.45 


0.12 


0.05 


-0.10 


0.00 


-0.02 


0.18 


0.01 


math/logic 


0.19 


-0.09 


0.51 


0.01 


0.12 


0.06 


0.14 


-0.08 


math/logic 


0.09 


-0.06 


0.18 


0.00 


0.07 


0.17 


0.26 


-0.10 


math/logic 


0.75 


0.03 


0.00 


-0.04 


-0.04 


-0.04 


0.04 


-0.01 


math/logic 


0.73 


-0.07 


-0.05 


-0.06 


0.12 


0.04 


-0.06 


0.06 


math/logic 


0.31 


0.16 


-0.02 


-0.11 


-0.02 


-0.04 


0.33 


0.12 


spatial 


0.13 


0.09 


0.10 


0.08 


-0.09 


-0.15 


0.31 


0.03 


spatial 


0.04 


-0.01 


-0.01 


0.04 


0.04 


-0.12 


0.62 


0.05 


spatial 


-0.02 


0.07 


-0.01 


0.01 


-0.12 


-0.04 


0.77 


-0.02 


spatial 


0.16 


0.02 


0.02 


0.02 


0.21 


0.04 


0.15 


0.34 


spatial 


0.21 


0.06 


0.06 


0.08 


0.04 


0.02 


0.07 


0.63 


spatial 


0.38 


-0.05 


0.04 


0.06 


0.03 


0.09 


0.05 


0.52 


spatial 


0.23 


-0.13 


0.09 


-0.01 


0.12 


-0.06 


0.43 


0.32 


spatial 


0.32 


-0.07 


-0.01 


0.05 


0.02 


-0.10 


0.39 


0.25 


spatial 


0.16 


-0.09 


0.05 


0.00 


-0.05 


-0.03 


0.58 


0.12 


spatial 


-0.11 


-0.06 


0.07 


0.08 


0.03 


0.11 


0.59 


-0.09 


spatial 


-0.09 


0.11 


0.13 


-0.06 


-0.05 


0.04 


0.57 


-0.02 


spatial 


0.20 


0.00 


0.08 


0.01 


0.12 


0.07 


0.12 


0.59 


spatial 


0.12 


-0.08 


0.09 


-0.02 


0.40 


0.06 


0.15 


0.26 


spatial 


0.19 


-0.02 


0.01 


0.00 


0.02 


0.10 


0.42 


0.01 


spatial 


-0.02 


0.02 


0.00 


0.01 


-0.06 


0.12 


0.71 


-0.04 


linguistic 


-0.15 


0.09 


-0.02 


0.10 


-0.12 


0.59 


0.01 


-0.09 


linguistic 


-0.12 


-0.05 


0.01 


0.24 


-0.05 


0.47 


0.06 


-0.04 


linguistic 


-0.10 


-0.02 


0.00 


0.08 


-0.09 


0.66 


0.06 


-0.03 


linguistic 


-0.06 


-0.10 


-0.02 


0.29 


-0.07 


0.51 


0.06 


-0.05 


linguistic 


0.03 


0.05 


-0.03 


-0.02 


0.19 


0.63 


-0.09 


0.14 


linguistic 


0.04 


0.16 


-0.05 


-0.09 


0.12 


0.47 


. 0.04 


0.15 


linguistic 


-0.04 


0.19 


-0.02 


-0.07 


0.09 


0.52 


0.05 


0.13 


linguistic 


0.05 


0.07 


0.03 


-0.02 


0.19 


0.51 


-0.10 


0.05 


linguistic 


0.05 


0.24 


-0.03 


-0.04 


0.20 


0.39 


-0.03 


0.09 


linguistic 


-0.05 


0.13 


-0.02 


-0.02 


0.08 


0.60 


-0.19 


-0.04 


linguistic 


0.04 


0.18 


-0.01 


-0.07 


0.10 


0.53 


-0.07 


0.03 


linguistic 


0.00 


-0.05 


0.03 


0.09 


-0.17 


0.57 


0.00 


-0.07 


linguistic 


0.01 


0.15 


-0.06 


0.00 


0.06 


0.56 


-0.03 


0.12 


linguistic 


-0.24 


-0.04 


-0.05 


0.09 


0.11 


0.44 


0.24 


0.05 
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linguistic 

linguistic 

linguistic 

linguistic 

linguistic 

linguistic 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

interpersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

IntrApersonal 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 

Naturalist 



0.08 


0.00 


0.01 


0.05 


0.04 


0.04 


0.04 


0.23 


0.11 


-0.14 


0.03 


0.06 


-0.02 


0.57 


-0.07 


0.63 


0.12 


0.38 


-0.11 


0.34 


-0.03 


0.65 


-0.03 


0.59 


-0.09 


0.57 


-0.03 


0.41 


-0.13 


0.59 


-0.02 


0.43 


0.04 


0.52 


0.00 


0.42 


-0.11 


0.53 


0.04 


0.54 


0.12 


0.22 


0.01 


0.34 


-0.07 


0.44 


-0.06 


0.38 


0.14 


0.37 


0.09 


0.42 


0.19 


0.38 


0.23 


0.37 


0.15 


0.49 


0.22 


0.42 


0.10 


0.19 


-0.06 


0.22 


-0.04 


0.24 


-0.11 


0.12 


-0.10 


0.19 


0.01 


-0.03 


0.01 


-0.01 


-0.03 


0.01 


-0.06 


0.02 


0.05 


-0.09 


-0.01 


0.02 


0.02 


-0.05 


0.24 


-0.19 


-0.01 


0.07 


-0.06 


0.02 


-0.12 


0.08 



0.01 


0.00 


0.00 


0.02 


0.03 


0.04 


0.02 


-0.01 


0.11 


0.03 


-0.02 


-0.01 


-0.06 


0.04 


-0.01 


-0.01 


-0.11 


0.01 


0.02 


-0.02 


0.00 


0.05 


0.01 


0.05 


-0.03 


-0.02 


0.01 


-0.08 


-0.04 


0.01 


0.02 


0.08 


-0.02 


-0.01 


0.06 


0.07 


-0.10 


0.10 


0.04 


0.01 


0.03 


-0.07 


-0.02 


-0.02 


-0.07 


0.03 


0.00 


-0.02 


0.03 


-0.09 


0.05 


-0.11 


0.01 


-0.08 


0.05 


-0.05 


0.05 


-0.04 


0.03 


0.00 


0.10 


0.03 


0.02 


0.05 


0.05 


0.00 


0.64 


0.04 


0.67 


0.00 


0.78 


-0.03 


0.65 


-0.01 


0.75 


0.03 


0.81 


0.01 


0.74 


0.00 


0.56 


-0.05 


0.68 


-0.04 


0.48 


-0.02 


0.63 


0.01 


0.50 


0.08 


0.64 


0.01 



-0.13 


0.65 


-0.16 


0.60 


- 0.30 


0.48 


-0.23 


0.12 


-0.13 


0.66 


-0.03 


0.69 


0.01 


-0.06 


-0.02 


0.07 


0.13 


0.34 


0.07 


0.00 


-0.10 


-0.03 


-0.13 


-0.05 


-0.09 


0.06 


0.05 


0.17 


-0.04 


0.13 


0.02 


0.00 


0.05 


0.09 


-0.08 


-0.09 


0.00 


-0.06 


0.09 


-0.13 


0.09 


0.23 


-0.04 


0.04 


-0.06 


-0.15 


0.06 


0.19 


0.03 


0.13 


-0.01 


0.06 


-0.01 


0.08 


0.02 


0.13 


0.03 


0.02 


0.05 


0.03 


-0.13 


-0.06 


-0.08 


0.04 


0.02 


0.11 


0.00 


-0.04 


-0.02 


-0.09 


0.10 


-0.02 


0.03 


-0.08 


0.00 


0.01 


0.00 


0.02 


-0.01 


0.03 


-0.06 


-0.04 


-0.09 


0.04 


0.10 


0.09 


0.03 


-0.04 


-0.02 


0.05 


0.03 


0.04 



-0.04 


-0.07 


-0.02 


0.00 


-0.03 


-0.20 


-0.03 


-0.18 


-0.07 


-0.02 


-0.06 


0.05 


-0.02 


0.04 


-0.03 


0.00 


-0.03 


0.04 


-0.09 


-0.01 


0.03 


-0.07 


0.16 


-0.02 


0.12 


-0.02 


0.08 


0.04 


0.08 


0.00 


-0.07 


0.02 


-0.06 


0.09 


0.04 


-0.03 


-0.06 


0.02 


-0.08 


-0.04 


-0.13 


-0.09 


0.00 


0.06 


-0.13 


0.00 


0.09 


0.06 


0.00 


-0.12 


0.08 


-0.07 


0.07 


-0.24 


0.02 


-0.08 


-0.10 


-0.04 


0.11 


-0.15 


-0.04 


0.01 


0.16 


-0.25 


0.10 


0.03 


-0.15 


0.11 


-0.15 


0.07 


-0.04 


0.08 


-0.02 


0.05 


-0.10 


0.13 


-0.09 


0.12 


0.09 


-0.01 


0.17 


-0.03 


0.11 


-0.03 


0.11 


-0.01 


0.10 


-0.14 


0.25 


-0.16 


0.09 


-0.23 



Note: This analysis yielded a total matrix sampling adequacy ratio of .946 and an average variable complexity ratio 
of 1.77. 



The item loadings for factors representing Musical, Linguistic, Interpersonal, and Naturalistic intelligences 
were fairly robust. Loadings for items representing Physical, Math/logic, Spatial, and Intrapersonal scales were not 
clearly associated with any single factors. The Intrapersonal scale loaded on the same factor as the Interpersonal 
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scale, indicating that the two constructs are not adequately distinguished from each other. The Spatial scale items 
loaded on two separate factors suggesting that the items may be measuring two different constructs. Additionally, 
some items double loaded on more than one factor and a few items did not load strongly with any specified factor. 

2 nd Order EFA. Due to this relative promiscuity of factor loadings on the variables, a 2 nd order EFA to 
explore the possibility of higher order constructs was conducted. Using a root curve extraction, four factors were 
produced (Obique rotation) with eigenvalues of 1 .3, .98, .72, and .54 respectively accounting for .44 of the variance. 



Table 3. 2 nd Order EFA, Oblique Solution, 4 Factors 





Factor 1 


Factor 2 


Factor 3 


Factor 4 


Obi 1 


-6.76E- 05 


0.955 


-0.033 


0.019 


Obi 2 


-0.109 


-0.036 


- 0.443 


-0.054 


Obi 3 


-0.26 


-0.113 


-0.059 


-0.069 


Obi 4 


5.37E-05 


-0.023 


-0.016 


0.731 


Obi 5 


-0.075 


-0.13 


0.154 


0.046 


Obi 6 


.001 


-0.005 


0.743 


-0.181 


Obi 7 


.932 


-0.148 


-0.126 


-0.183 


Obi 8 


0.026 


-0.057 


0.076 


0.011 



Conclusions 

Reliability coefficients of all scales were .85 - .90, greater than .70 are considered adequate (Kline, 1993). This was 
consistent with the authors previously reported results on other smaller samples. The internal scale structure as 
represented by the eight factor solution yielded by the exploratory factor analysis generally supported the theoretical 
framework and the intent of the instrument’s scales (Kline, 1993; Thurstone 1947) for five of the scales (Musical, 
Math/Logic, Linguistic , Interpersonal, and Naturalist) . The other scales were not clearly defined. Promiscuity 
among the variables items was evident with multiple loadings on some scales (Spatial and Physical). This begged 
for a follow-on second order in an attempt to make sense of possible higher order constructs and the MIDAS’s 
theoretical interrelationships. The 2 nd order on this sample did not assist in an additional theoretically relevant 
structure to ponder. 

This instrument represents an incomplete attempt to develop a valid assessment of “multiple intelligences.” 
Further work refining conceptually some of the subscales may lead to more robust measures of these constructs. We 
recommend that additional EFA’s and separate confirmatory models (CFAs) be conducted to further test the 
instrument’s psychometric properties and theoretical framework after modifications are made. Additionally, 
construct validity studies are also needed comparing the scales to other external constructs or instruments. Finally, 
practitioners and users of the MIDAS should be aware that this instrument, like many others used in the human 
resource development arena is based on self report data. The MIDAS could be useful as an initial diagnostic tool, 
but is not by itself a measure of aptitude or intelligences without observation of behaviors. Arguably, there should 
be some correspondence between honest self -assessment and actual measure of aptitude for these dimensions. 
When used appropriately, for example in a career exploration, career development, or group development context, a 
valid instrument assessing relative strengths in multiple intelligences could have great utility. 
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Evaluating Corporate Quality of Life Programs: Utilizing the Balanced Scorecard Model 
in Conjunction with Quasi-Experimental Design 



Wendy J. Casper 
Ann L Landy 
Caliber Associates 



To help a large, decentralized government agency comply with the Government Performance and 
Results Act, a performance monitoring system was developed to assess the effectiveness of quality 
of life programs — Child and Youth Services, Fitness and Wellness programs, Food Service, and 
Family Support Programs . This paper discusses the applicability of the Balanced Scorecard 
model and preliminary results of a quasi-experimental design for measuring the effects of the 
programs on s related to corporate performance outcomes . 

Keywords: Quality of life programs, Balanced Scorecard, Performance Measurement 

Research — primarily with private sector organizations — indicates that work and quality of life (QOL) have 
reciprocal relationships (Crosby, 1991). Private sector "work-life" programs are designed to help employees 
achieve balance between their work and their lives outside work (Friedman & Galinsky, 1992). Government 
organizations have the same work-family balance issues and have implemented similar programs. Given changes in 
the nature of work (i.e. employment relationships, workforce demographics), QOL issues are highly salient today 
and quality of life has become an increasingly important research topic. Moreover, since the implementation of the 
Government Performance and Results Act, in 1993, government employers are now compelled to assess the 
effectiveness of their operations, including their QOL programs. This paper describes the development of a 
performance monitoring system for the QOL programs of a large, decentralized government agency. The system 
was designed using the Balanced Scorecard (BSC) model for strategic planning, measurement and program 
management in conjunction with a synthesis of research findings. It will provide ongoing measurement of 
administrative efficiency and impact on corporate performance outcomes such as recruitment, retention, absenteeism 
and customer satisfaction. 

Research Questions 

This paper examines three questions: 

■ Can the Balanced Scorecard approach be used to create a workable system for measuring the performance 
of QOL programs? 

■ Can such a system deliver evidence of the contribution of QOL programs to corporate outcomes? 

■ If so, which corporate outcomes (satisfaction, retention, recruitment, absenteeism, productivity) are 
influenced by which QOL programs (childcare, food service, fitness and wellness, family programs)? 

Background 

The Balanced Scorecard (BSC) — a model for strategic planning, measurement, and program management — was 
adopted by the government agency for all of its corporate support programs and offered a promising approach for 
conceptualizing, defining, and implementing the QOL performance monitoring system. Developed by Robert 
Kaplan and David Norton (1996), the model defines a procedure for establishing performance goals and using 
multiple measures to make management decisions. As a four-phase management tool (see Figure 1), the BSC 
allows managers to ensure that business decisions are not based solely on financial data. It also allows some 
measures to influence decisions more than other measures by weighting the data. While the model should not be 
construed as a theory of performance measurement, a measurement system defined from this framework is an 

Copyright © 2001 Wendy J. Casper and Ann L. Landy 
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implicit logic model for predicting the impact of the QOL programs as interventions. The BSC approach involves 
program staff in developing a performance measurement system that tailors the measurement process closely to 
program activities. Moreover, based on the notion that multiple measures provide more explanatory power, the 
model incorporates customer satisfaction, program management, and cost effectiveness data as well as corporate 
outcome measurement, all of which were identified by agency staff as critical performance domains. 

Methodology 

The diagram in Figure 1 illustrates the way the BSC functions as a strategic framework. It shows a cycle of four 
activities for which the scorecard is the central or anchoring element. The development of the QOL business plan 
and performance monitoring system was consistent with the BSC development model (see Figure 1). In the first 



Figure 1. 

The Balanced Scorecard as a Strategic Framework 
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phase the QOL vision was clarified and consensus was gained about design issues. Meetings were held with 
program managers over a twelve-month period to (1) review background research and data, (2) define a strategy for 
developing the business plan, (3) develop the mission and vision statements, and (4) define goals and objectives. 
These statements formed the foundation of the business plan and performance monitoring system. An effort was 
made to define a few discreet goals and objectives for which achievement could be measured and that would lead to 
program improvement. Three major goals were defined, and three objectives were defined for each goal. The goals 
and objectives comprise the framework for performance monitoring and program development. 

Phase I: Clarifying and Translating the Vision and Strategy 

The first phase was devoted to clarifying the QOL vision and gaining consensus among program managers 
about system design issues. Considerable effort was invested in preliminary research to acquire the information 
necessary to help managers define their expectations about program effects and the types of measures needed to 
demonstrate results. Conprehensive coverage of background issues involved: (1) reviewing the literature QOL 

programs in comparable organizations to determine where corporate outcomes had been affected by QOL programs 
and which QOL programs could affect corporate outcomes; (2) conducting a survey of supervisors to determine 
their beliefs about QOL program effects on corporate outcomes and; (3) conducting focus groups at several 
locations to assess perceptions about program strengths and weaknesses which augmented the empirical findings. 




37-2 




The findings — described below — helped managers identify feasible and potentially sensitive measurement 
procedures. 

Child and Youth Services . This organization offered a variety of child and youth programs including full 
day on-site childcare, before and after school childcare, and childcare vouchers. Studies that examined the impact of 
on-site childcare on attitudinal outcomes reported mixed results. One literature review found little evidence that on-site 
childcare enhanced job satisfaction (Miller, 1984) while other research found an increase in organizational commitment 
and job satisfaction with the presence of on-site childcare (Kossek & Nichol, 1992; Youngblood & Chambers-Cook, 
1984). In addition, while one study found access to on-site childcare resulted in the perception of greater work- 
family balance (O’Brien, Maher, Buffardi, Smith, & Erdwins, 1997), other research found this benefit was unrelated 
to work-family conflict (Goff, et al, 1990). Research examining the impact of on-site childcare on behavioral 
outcomes (i.e. absenteeism) provided less support for its efficacy. Though one study (Youngblood & Chambers- 
Cook, 1984) found a small reduction in absenteeism from 8% to 3% with implementation of on-site childcare, this 
change was not statistically significant. Kossek and Nichol (1992) found that supervisors did not rate users of on- 
site childcare differently than non-users with respect to performance or perceived absenteeism. Finally, Goff, et al 
(1 990) found that presence of on-site childcare did not reduce absenteeism. Likewise, focus group members felt that 
unless sick childcare was offered (it was not), on-site childcare was unlikely to influence absenteeism. 

Fitness and Wellness Programs. The agency offered on-site fitness centers as well as a variety of wellness 
education seminars on topics such as smoking cessation, nutrition, and stress management. The literature included 
studies of programs such as on-site fitness centers, nutrition education programs, health promotion programs, fitness 
and health risk evaluations, and supervised aerobic classes. Overall, empirical research on fitness and wellness 
programs provided some support for the impacts of such programs on important outcomes. One study (Leutzinger 
& Blanke) explored the effects of fitness centers on perceived productivity and fitness perceptions, and found that 
fitness center members were more likely than non-members to report positive impressions of the center and beliefs 
that use was linked with worker productivity. Another study (Sanderson, 1986) found that participation in an 
employee fitness program was related to positive health outcomes, increased job satisfaction, and decreased 
tardiness. Likewise, Gallant (1986) found that participation in an employee health promotion program was 
associated with fewer health risk factors, increased job satisfaction, and decreased absenteeism. Bertera (1993) 
found that at-risk employees who participated in a health promotion program used less sick leave and had more 
favorable results on a variety of health indicators such as cholesterol level, systolic blood pressure, amount of 
alcohol consumed, and use of seat belts. Finally, a study of the effects of stress management training (Leakey, 
Littlewood, Reynolds, & Brunce, 1994) found that participation was related to an enduring increase in psychological 
well-being. 

Food Service Programs. While the anecdotal literature on QOL programs offers examples of employers 
who offer on-site cafeterias and take-out meals to their employees, no published empirical research evaluating these 
programs has been conducted. However, since this organization offered food services as a core QOL program, they 
were interested in exploring its impact on corporate outcomes. In addition, focus group results revealed that QOL 
managers perceived on-site food service was linked to increased satisfaction, shorter lunch breaks, and increased 
worker productivity. 

Family Support Programs. The family support programs in this organization included both relocation 
assistance for relocating employees and their families, and a Family Advocacy Program (FAP) that provides 
services aimed at prevention and treatment of family problems through programs such as anger management and 
effective parenting. The extant empirical literature had not explored the effects of these types programs on 
corporate outcomes. However, as core programs for this agency, they were included in the evaluation plan. 
Furthermore, program managers who participated in focus groups expressed their beliefs that relocation assistance 
services were associated with decreased stress, lower absenteeism, and enhanced financial well-being, and that FAP 
use was related to increased satisfaction. 

Phase 2: Communication: Linking and Setting Goals 

The second activity involves communicating information about the strategic objectives, performance 
weighting, and the BSC to all business units, obtaining commitment from the staff, and helping each unit set long- 
term (“stretch”) goals. Long-term (three- to five-year) goals are used to plan, set targets, define milestones, and 
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align strategic initiatives. In this phase, specifications for goals and objectives were presented to field staff for 
review. The presentation also was an opportunity to describe the advantages of having a homogeneous framework 
for assessing performance and the projected use of the data to support budget requests. Participants provided 
valuable information about design shortcomings, implementation obstacles, and data collection issues, all of which 
were incorporated in revisions to the design. 

Phase 3: Planning and Target Setting 

The third activity required that field staff for each QOL program area define short-term (one-year) goals, 
measures, minimum acceptable performance levels, and a weighting system (point allocation for each measure) 
appropriate for their operation. While overall goals and objectives were developed to apply across all four program 
areas, performance standards and indicators had to be program-specific. Using the goals and objectives as a 
framework, field staff members reviewed draft lists of standards, indicators, and measures, and revised material 
based on the utility of the information and feasibility of collecting the data. Thus, targets were established, staff 
gained an understanding about how program-specific strategic initiatives could align with goals and objectives (e.g., 
for marketing efforts), resources were identified, and milestones were established. Here, the procedure departed 
from the BSC model and the illustration in Figure 1. Weights have not yet been assigned for core program 
contribution to overall QOL performance or for each type of data to core program performance. Weights can be 
assigned at the end of the business cycle when the first round of data collection is complete and results are reviewed. 
This staff review was important to implementation. Program staff members now have a common frame of reference 
for measuring performance, but they define standards and measures that reflect the specific function of their 
program. They also play a key role in defining the standards by which the program will be judged and possess clear 
understanding of the data collection procedures. 

Phase 4: Strategic Feedback and Learning 

The fourth activity involves the use of scorecard data to assess progress and make business decisions. As 
the diagram in Figure 1 suggests, this process feeds directly back into the clarification and translation activity and 
sets system modifications in motion for the next cycle. The “scorecard” includes “scores” on multiple measures of 
performance in several categories, such as financial data, customer feedback, internal management indicators, and 
measures of employee skills and skill development needs. “Balance” is achieved by applying weights to 
performance data that reflect judgments about importance. For example, financial data may be weighted more 
heavily than data on addressing employee development needs. In the BSC model, balance also should be achieved 
between external measures (e.g., for customers and stakeholders) and internal measures or “performance drivers” 
(e.g., measures of business processes), as well as between quantitative data (e.g., outcome measures such as net 
profit or average response time) and qualitative data (subjective judgements). 

Figure 2 illustrates how the organization can use the BSC approach. The table includes performance data, 
the performance targets to which the data are compared, and the use of weights to generate scores. Scores are 
tabulated for four categories of outcomes: corporate outcomes, program management, cost performance, and 
customer satisfaction. Of the performance categories, customer satisfaction was judged by the childcare staff to be 
most important. Scores for customer satisfaction receive a weight of 30 of 100 points. Of the four customer 
satisfaction measures, parent ratings of curriculum are considered most important. The on-site childcare business 
unit can earn a maximum of 1 0 of 30 points if performance targets are reached. 

Quasi-Experimental Research Design 

The quasi-experimental design was integrated into the BSC approach through the examination of corporate 
benefit. In designing research to explore the effects of QOL programs, the dilemma was to generate the most 
rigorous design possible within this applied setting. While the research design providing the greatest degree of rigor 
is the randomized experimental design (Cook & Campbell, 1979), this design requires (1) to have both an 
experimental (program participation) and a control (no participation) group, and (2) random assignment to 
conditions. Given the pragmatic constraints of the organization, random assignment was not possible. Thus, the 
system incorporates a quasi-experimental design for data analysis that includes treatment and control groups, but 
without random assignment. This quasi-experiment will be used to examine the impact of QOL programs on 
outcomes. For each program area, users and non-users will be compared on outcomes. In addition, the non-user 
comparison group will be chosen so that relevant demographic characteristics are similar to those in the user group. 
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For instance, when comparing users and non-users of on-site childcare, non-users will be chosen who are custodial 
parents of children of a similar age, but who use off-site childcare. Thus, when satisfaction levels and retention rates 
for the two groups are compared, findings will not be confounded by individual differences like need for childcare 
or age of children. Likewise, when satisfaction data from users and non-users of fitness center are compared, non- 
users will be matched to users on demographic characteristics, such as age, gender, and job type. 

Specific Hypotheses 

The literature review findings and focus group results were utilized to generate specific hypotheses 
regarding the impact of each of the four core programs on corporate outcomes. Performance standards are a 
fundamental component of a BSC. Defining performance standards is similar to stating predictions about what will 
be achieved as a result of implementing programs. With respect to QOL programs, predictions pertain to four 
categories of outcomes: customer satisfaction, management effectiveness, corporate benefit, and return on 
investment. Performance standards and measures were defined for each program area. Performance data will 
indicate whether the standards are met. The hypotheses are delineated in the section that follows. 

Child and Youth Programs. Based on the review of the childcare literature (Kossek & Nichol, 1992; 
Youngblood and Chambers-Cook, 1984), we expected that users of on-site childcare services would report more 
satisfaction with the organization than non-users who used off-site childcare. While empirical research did not 
examine effects of childcare on turnover, focus groups revealed perceptions that such services facilitated retention. 
In addition, it is reasonable that employees who have their children happily situated in on-site childcare would be 
more reluctant to change jobs. Thus, the following hypotheses were made regarding on-site childcare: 

H ypothesis 1 . Users of on-site childcare programs will exhibit greater satisfaction with the organization 
than employees who use off-site services. 

Hypothesis 2 . Users of on-site childcare programs will exhibit lower turnover than employees who use off- 
site services. 

Fitness and Wellness Programs. Several studies in the literature indicated fitness and wellness program 
use was linked to increased job satisfaction (Sanderson, 1986; Gallant, 1986) and reduced absenteeism (Gallant, 
1986). Thus, the following hypotheses were made regarding the effects of fitness and wellness programs: 

Hypothesis 3 . Users of fitness and wellness programs will be more satisfied with their jobs than non-users. 

Hypothesis 4 . Users of fitness and wellness programs will exhibit lower turnover than non-users. 

Food Service Programs r While no published research explored the effects of food service programs on 
corporate outcomes, results of our focus groups revealed that program managers expected a link between food 
service use and organizational satisfaction. Thus, the following hypothesis was generated: 

Hypothesis 5. Food service use and satisfaction will be positively correlated such that more frequent use 
of food services will be associated with greater satisfaction with the organization. 

Family Support Programs. While published research did not explore the effects of relocation services on outcomes, 
focus group participants felt individuals who used relocation assistance would exhibit less stress and spend fewer 
hours handling moving-related activities than those who did not. In addition, while published literature did not 
explore effects of FAP, focus group participants believed participation was linked to greater satisfaction. Thus, the 
following hypotheses were generated with respect to family support programs: 

Hypothesis 6 Employees who use the relocation assistance program will report lower stress than 
employees who relocate but do not use the program. 

Hypothesis 7 . Employees who use the relocation assistance program will report fewer hours away from 
work handling moving-related errands than employees who relocate but do not use the program. 

Hypothesis 8 . Employees who use FAP will exhibit higher satisfaction with the organization than 
employees who do not. 

Results 

The basic BSC development and implementation process involves four steps: (1) defining overall organizational 
performance goals and relevant indicators for measuring goal achievement, (2) communicating information about 
goals and measures to work units, (3) establishing performance targets and measurement procedures within units, 
and (4) using measures (performance data) to determine if goals have been achieved and re-define performance 
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Figure 2. Hypothetical Scorecard: On-site Childcare 
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goals and indicators. Currently, the first three steps have been completed and the fourth step will be conducted after 
data collection is completed. Used in conjunction with a thorough review of existing research, the approach 
functions well for developing a performance measurement system. It provides a common framework for developing 
performance objectives and for reporting results that can be tailored to individual program areas. Agency leaders 
believed the BSC would provide a common focus for the organization and it proved to be effective for establishing 
goals and objectives that apply to different programs and accommodate program-specific performance standards and 
data collection. However, this approach is not without shortcomings. Development required intensive participation 
of agency management and field staff. Agency QOL program mangers dedicated numerous hours to development of 
the business plan over a 12-month period. The managers participated in six day-long work sessions, reviewed 
materials independently, and facilitated the review by field staff. Moreover implementation of a BSC requires 
commitment of resources for planning and training. The required level of investment was achieved because the 
QOL program director supported the effort by setting expectations for staff and providing resources needed to fund 
contractor activities, hold work sessions, and provide materials. When fully implemented, a BSC data management 
system will permit on-going performance tracking from all levels of the organization and provides diagnostic 
information to address problems. When this agency’s data management system is implemented, performance data 
will facilitate tracking progress by program staff at multiple locations. Data can also be used to modify program 
plans. Thus, a BSC system can foster communication between levels and across functional areas in an organization. 
Achieving this enhanced communication is not easy. This project required hiring an external contractor to develop 
and maintain a database, requiring substantial time and financial resources. In addition, skilled IT staff who 
understand the difficulties in combining data from several electronic sources of different configuration are 
necessary. 

Conclusions and Recommendations. 

Despite some evidence that QOL programs contribute to corporate outcomes, literature review findings were 
inconsistent. A recurring criticism was that measures were not tied to program activities and were not developed by 
program staff that could inform predictions. These shortcomings were addressed by applying a BSC approach that 
requires intensive involvement of staff in the specification of goals, objectives, and measures. However, although 
the BSC model appears to be relatively simple, developing and implementing a BSC management system is a 
complex, labor-intensive process in which numerous obstacles can arise. First, the time required for developing and 
implementing a BSC system is much more than Kaplan and Norton’s 16-week estimate. Interruptions of several 
months occurred in the development of the QOL performance monitoring system. Nevertheless, three years were 
required to complete preliminary research and work through the design process. Additional time will be required to 
develop the necessary database. Second, the BSC requires an elaborate, complex data management system to 
deliver tracking and diagnostic information that foster communication. The complexities of database development 
are gradually becoming apparent as the implementation process begins and more development time is required than 
was anticipated. Third, there are several issues the BSC literature does not address. There is no discussion of 
instrument development and data management procedures. While these issues are being resolved in the current 
effort, it required more resources and time than anticipated. In addition, the BSC literature gives no guidance about 
achieving consensus on performance targets, appropriate measures, and data interpretation procedures. Achieving 
consensus requires skilled facilitation and staff determined to define a simple, workable system. Finally, data 
management is potentially the most serious obstacle to use of the BSC. Organizations contemplating a BSC 
management system should be aware of the data management issues so that they can anticipate cost implications 
before embarking on development. To meet data management requirements for this agency, a team of IT 
consultants will build an SQL-based data warehouse capable of integrating data for four program areas and five data 
sources. Furthermore, to make the BSC data management system accessible from computers at any level in the 
agency, and to allow employees at all levels to track performance, many security management issues need to be 
dealt with. 

Effects on Corporate Outcomes 

Conclusions cannot yet be drawn regarding the effects of QOL programs on corporate outcomes. However, 
once data are collected and analyzed we should be able to make data-based program recommendations. The hope is 
that QOL programs will be associated with greater retention, increased satisfaction, reduced stress, and reduced 
absenteeism. Program effectiveness at producing such outcomes is an important indicator of success. If, in fact, 
such programs are linked to corporate outcomes, this provides a strong argument for continued funding. However, 
if not linked to outcomes valued by the organization, it becomes more difficult to justify the resources necessary to 
maintain and grow them. Thus, while implementation of such a BSC management process is costly in time and 
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money, it may be more costly not to evaluate. Information obtained from this performance management system is 
critical in making effective program decisions, and ensuring program offerings facilitate the achievement of the 
organization’s goals. 

Contributions to HRD 

This paper describes the development of a performance monitoring system created by integrating the BSC approach 
and a quasi-experimental design to evaluate Quality of Life programs. Program evaluation is an important topic in 
HRD, given the usefulness of evaluation in demonstrating the importance of HRD activities to upper management. 
Yet the implementation of thorough evaluation programs remains a challenging aspect of HRD practice. While 
clearly demonstrating programs facilitate learning, change behavior, and lead to desirable organizational results is 
superior evaluation to asking participants if they like the programs, in reality most organizations still use brief 
reaction forms as their sole means of program evaluation (Bassi, Benson, and Cheney, 1 996). These circumstances 
have prompted diverse reactions from researchers from promoting the abolition of participant reactions (Holton, 
1996) to suggesting empirical work to enhance the development of reaction forms which might be more useful to 
organizations (Morgan and Casper, in press). While HRD practitioners often find themselves in challenging 
predicaments in which they must show program effectiveness, yet have little financial resources to do so, for 
programs to remain in spite of downsizing and cost-cutting, we must demonstrate their worth. For public sector 
employers, the need for thorough evaluation may perhaps be greater. Not only is it desirable for government 
agencies to go beyond reactions to evaluate programs, it is now mandatory if agencies are to comply with the 
Government Performance and Results Act. In response, many agencies are jumping on the Balanced Scorecard 
bandwagon. This paper demonstrates that while the Balanced Scorecard is clearly one useful approach to 
evaluation, (1) there are numerous implementation challenges that need to be managed, and (2) it is a more 
comprehensive evaluation when integrated with traditional forms of research design. In providing discussion of 
developing and implementing a BSC evaluation system, this paper serves as a tutorial for setting up a BSC 
evaluation, as well as alerts evaluators potential challenges to implementation of such an approach. 
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This study represents an extension of work to determining customer service competencies 
undertaken by Russ-Eft, Berrey , Boone, and Winkle (2000). The present study used the critical 
incidents that formed the basis of the previous work and the resulting competencies to develop and 
validate a tool to assess service competencies. Data were gathered from 885 customer service 
personnel in 10 different organizations. Results from the factor analysis and reliability analyses 
are presented and discussed. Recommendations concerning the role of rigorously researched 
instruments. 

Keywords: Customer Service, Competency Assessment, Instrument Validation 

Problem Statement 

In a highly competitive global business climate, more and more organizations are recognizing the pivotal role of 
service in attracting and retaining customers (Hart & Johnson, 1999; Jevons & Pidgeon, 1999). Seeking to create a 
customer base that is not merely satisfied, but wildly enthusiastic and loyal, these organizations are eager to learn 
from research which behaviors and competencies make a difference in customer service providers. In this context, 
identifying and measuring customer service competencies within organizations becomes critical. Previously, Russ- 
Eft, Berrey, Boone, and Winkle (2000) investigated and identified a comprehensive set of service competencies. 
The present study describes the development and validation of an instrument designed to measure those 
competencies. 

Theoretical Framework 

Although both the practitioner and the academic literature have paid considerable attention to customer concerns, 
investigation of the competencies needed by customer service providers and others within organizations to win and 
maintain customer loyalty have drawn minimal attention. Models of customer service competencies appear or can 
be gleaned from the following: Berry, Parasuraman, and Zeithaml (1994), Bhote (1996), Bitner, Booms, and 
Tetreault (1990), Corcoran et. al. (1995), Learning International (1991), Levesque (1995), and Zemke (1995). A 
review of these models, all of which were developed in the late 1980s or early 1990s, reveals at least three 
competencies common across the various reports: (1) listening and communicating, (2) being reliable, consistent, 
and dependable, being respectful, courteous, and fair, and (3) solving problems. 

Note, however, that only the Berry et. al. (1994), Bitner et. al. (1990), Corcoran, Petersen, Baitch, & Barrett 
(1995), and the Learning International (1991) studies were based on primary data collection. The first used various 
focus groups and surveys of customers and interviews of customer service personnel; the second used critical 
incidents gathered from customers; and the latter two used interviews with managers and customer service personnel 
or sales personnel. 

As organizations enter the 21st century, decision-makers in organizations identified four business 
challenges facing customer service (Russ-Eft, et al, 2000): 

• The customer’s need for value balanced against the shareholder’s need for profit. 

• The customer’s need for special attention balanced against the manager’s need for productivity. 

• The organization’s need to appeal to more “profitable” customers balanced against its commitment to 
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serve all customers 

• The customer’s new demands balanced against the organization’s ability or willingness to meet those 
demands. 

In light of the need for a current examination of customer service competencies, Russ-Eft et al (2000) 
undertook and reported on such a study involving over 2,100 critical incidents, gathered from frontline service 
representatives, others in the organization, and customers. Five major customer service factors emerged from the 
analysis The resulting SERVE™ model provides a comprehensive view of service behaviors and beliefs: 

■ See the “big picture” and how customer service fits into it -•< 

■ Establish an authentic human connection with each customer 

■ Render timely, accurate and thorough service 

■ Value and respond to unique customer needs 

■ Extend a hand to repair and strengthen relationships with customers who are upset or angry. 

Figure 1 presents a description of the five factors. 



Competency 



Description 



See the “big picture and 
how customer service fits 
into it. 

Establish an authentic 
human connection with 
each customer. 

Render timely, accurate 
and thorough service. 



Value and respond to 
unique customer needs. 



Extend a hand to repair 
and strengthen relation- 
ships with customers who 
are upset or angry. 



Understands organizational, customer, personal/job consequences; 
helps, trains colleague; compensates for colleague’s error, failure or 
absence; fixes product or system; develops new system. 

Avoids phone frustration; calms upset customer; makes customer feel 
valued; acts with patience; provides friendly, courteous service. 



Gathers information; listens; helps customer through the process; 
thoroughly explains services/procedures; keeps customer informed; 
follows up; solves problem; finds alternate product or service; 
expedites request; takes action that was the responsibility of other 
departments; provides complete and detailed service. 

Bends rules involving information, price or fees, procedures, 
warranty/insurance; accommodates special need or request; responds 
to urgent request; works off-hours to complete request; gives extra 
service beyond those contracted; does a kind act; fixes a customer’s 
mistake; provides extras. 

Recovers by speeding up process, payment/price accommodation, 
giving special attention, fixing the problem; replacing product/service, 
having manager solve the problem; goes outside organization to help 
customer; calls another organization to complete own job; helps 
confused customer who called the wrong organization; refers customer 
to competitor or other service. 



Figure 1 . Description of Customer Service Competencies 



Recently the Job Analysis and Competency Modeling Task Force, a work group jointly sponsored by the 
Professional Practice Committee and the Scientific Affairs Committee of the Society for Industrial and 
Organizational Psychology, published a review of work in both competency modeling and job analysis. As part of 
its analysis, the Task Force set forth some recommendations for competency modeling and validated instruments. 
The Task Force created a level-of-rigor scale and described “high rigor” as characterized by “a variable combination 
and logically selected mix of multiple methods used” gchippmann et al, 2000, p. 716). Given this recent 
recommendation and its definition of level of rigor, it may be important to add additional quantitative analyses to 
supplement the qualitative analyses in order to assess the SERVE™ competency model. 
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Research Questions 



Having developed the competency model outlined above, we turned to the creation and validation of an instrument 
to measure these competencies. At this point, three key research questions were: 

■ What factors emerge from such an instrument? 

■ How do these factors correspond with the SERVE™ model? 

■ What is the reliability of such an instrument? 



Method 

Instrument 

The instrument developed for the present study was based on the SERVE™ model, and items were based 
on the critical incidents from the Russ-Eft et al (2000) study. Each item stated a behavior or a belief, such as “Greet 
each customer in a friendly manner” or “Stay calm when others are upset or angry.” Respondents used a seven- 
point Likert-type scale to indicate the extent to which they exhibit the behavior or hold the belief. A response of 1 
means “Not at all,” a response of 4 means “About 50% of the time,” and a response of 7 means “100% of the time.” 
Respondents were also asked how important they believed the behavior is to the effective performance of customer 
service. Here, 1 means “Not at all important,” 4 means “Somewhat important,” and 7 means “Very important.” For 
validation purposes, only the responses to the scale asking for ratings of “extent to which the respondent exhibits the 
behavior or holds a particular belief’ were used. 

Approximately 90 items were drafted. Content experts in customer service reviewed the items for content 
and clarity. Seventy-seven items were retained for use in this study. 

Subjects 

The participants in this study were employees from ten different organizations, covering a variety of 
industries including higher education, insurance, business services, heavy manufacturing, and wholesale trade. Due 
to the way the instruments were distributed, the researchers do not know how many employees received a 
questionnaire, however, 885 instruments were returned. Of these, the majority of the participants were women 
(65%). The age category checked most frequently was 26 to 35 (37%). In general, the participants were well 
educated. Only 1% had not completed high school; of the remainder, 19% were high school graduates. The 
majority of participants had some higher education, whether vocational/technical studies (42%), a bachelor’s degree 
(27%) or a master’s degree or higher (7%). Nearly three-fourths of the participants had had some previous training 
in customer service skills (73%). 

Procedure 

A coordinator within each organization was contacted by telephone and asked to distribute questionnaires. 
Typically, this coordinator was someone in the Human Resources or Training department of the organization.Each 
subject received a questionnaire and a self-addressed stamped envelope for returning it. 

Analyses 



A principal-components factor analysis was undertaken, and items were deleted based on cross-loadings. 
After these deletions, Cronbach’s alpha was calculated to determine internal consistency. Then an alpha “if item 
deleted” was calculated to determine the effect on the alpha when items were deleted from the scale. 

Results 

The total number of respondents was 885. Of these, 24 had 20% or more missing responses. These cases were 
removed from the analyses due to incomplete data. Thirteen participants indicated a response of 7 more than 90% 
of the time. Due to the lack of variance, this group was also removed from the analyses. 

Prior to conducting the analyses, the item responses were checked for normal distribution. Every item was 
significantly negatively skewed; that is, the responses tended to be clustered at the top end of the scale. In addition, 
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almost every item had negative kurtosis, meaning that the responses were clustered together more tightly than would 
be expected in normal distribution. The five items with the worst skew and kurtosis were removed from the 
analyses, reducing the number of items analyzed to 72. 

A visual inspection of the item response distributions revealed that, although the skew was significant, it 
was not extreme. Furthermore, the skew was in the same direction for every item. Given the relatively large 
sample size, nonnormality is unlikely to have an effect on the analyses. In addition, Waternaux (1976) states that, 
with positive kurtosis, underestimation of variance disappears with samples of 200 or more. 

Within the 848 cases included in the analyses, approximately 0.5% of the data were missing. A T-test was 
conducted to check for differences between respondents who answered every question and those who skipped at 
least one, but fewer than 15. No differences were found, (t = .312, df = 846, p = .755). This implies that there are 
no systematic differences between the two groups. Based upon that conclusion, missing data for an item was 
replaced with the overall mean for that item. Replacing the missing data allowed the retention of these respondents 
in the factor analysis. 

The final sample of 848 resulted in a respondent to item ratio of 1 1.8 to 1 . A ratio of between 5 to 1 and 10 
to 1 is recommended for undertaking a factor analysis (Hair, Anderson, Tatham & Black, 1995; Nunnally & 
Bernstein, 1994). 

Factor Analysis 

Prior to conducting factor analysis', the data set was tested for factorability. An inspection of Pearson’s 
correlations shows that there are numerous correlations above 0.3, indicating that factor analysis is appropriate for 
this data set. The data set also meets the criteria for the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO 
= 0.96, values of .6 and above are required for good factor analysis (Tabachnick & Fidell, 1996). A principal 
components factor analysis was conducted using the 72 remaining items from the questionnaire. Principal 
components is appropriate for this type of study, because it will provide a unique mathematical solution that may 
help determine the number and nature of the underlying components (Tabachnick & Fidell, 1 996). Since this study 
is the first use of this instrument, such an exploratory analysis is required. 

For the analysis, all factors with an eigenvalue greater than one were retained. An eigenvalue is a measure 
of variance accounted for by the factor. An eigenvalue of one means that the factor accounts for approximately the 
same amount of variance as an individual item in the analysis. In this analysis, there were fourteen factors with 
eigenvalues greater than one. An inspection of the scree plot yields no evidence thafwould indicate a different 
number of underlying factors. The total variance explained is 55%. 

To facilitate interpretation, the factors were rotated. An inspection of the factor correlations when the 
factors were rotated obliquely show that the factors were not substantially correlated with one another (only two 
correlations were above 0.3, and approximately two-thirds of the correlations were below 0.2). Therefore, an 
orthogon (Varimax) was used for the remaining analyses. An examination of the rotated factor loadings revealed 
several items that loaded on more than one factor, and furthermore, there were several factors with only one item 
with a major (>0.30) factor loading. To clarify the factors, this analysis was repeated several times, and each time 
some of the cross-loading items were eliminated. The final factor analysis consists of 49 items. As the number of 
items was reduced, the factors coalesced into nine factors. Table 1 presents the nine factors, the average loading on 
the major factors, the average loading on the minor factors, and the percentage of variance explained. The average 
loadings were calculated using the absolute value of the loadings. 

These nine factors can be mapped with accuracy to the competencies developed in the critical incident study. 
For example, two factors, Being a Customer Service Advocate and Supporting Your Coworkers) map onto the “S” 
competency of “See the “big picture and how customer service fits into it.” Providing Core Customer Service 
focuses on aspects of the “E “ competency, “Establish an authentic human connection with each customer.” Both 
Follow Through and Providing Information to Customers pertain to providing attentive service to customers and can 
be related to the “R” competency: “Render timely, accurate and thorough service.” Two factors, Beyond Core 
Customer Service and Dealing with Diverse Customers, mirror aspects of the “V” competency: “Value and respond 
to unique customer needs.” Coping with Difficult Customers/Situations fits with the “E” competency: “Extend a 
hand to repair and strengthen relationships with customers who are upset or angry.” In addition to these dimensions 
of the SERVE™ model, another factor emerged that was labeled Organizational Support for Customer Service. 



Table 1. Factor Analysis Results 



Factor 


Number of 
Items 


Avg. Loading - 
Major Factor 


Avg. Loading - 
Minor Factor 


% Variance 
Explained 


Providing Core Customer 
Service 


14 


.57 


.21 


13.4% 


Beyond Core Customer Service 


6 


.54 


.15 


6.5% 


Follow Through 


5 


.61 


.13 


5.8% 


Providing Information to 
Customers 


6 


.49 


.15 


5.6% 


Coping with Difficult 
Customers/ Situations 


4 


.56 


.15 


5.6% 


Being a Customer Service 
Advocate 


5 


.55 


.11 


4.8% 


Organizational 

Support For Customer Service 


4 


.52 


.12 


4.5% 


Dealing with Diverse 
Customers 


3 


.52 


.12 


3.8% 


Supporting Your Coworkers 


2 


.72 


.09 


3.2% 



Reliability 

After eliminating some items, as described above, and after determining the factors using the resulting 49- 
item instrument, the survey was measured for internal consistency among its factors using Cronabach’s alpha. 
Alpha is an indication of the proportion of the variance in a scale score that can be attributed to differences in the 
underlying construct. Table 2 presents these data. 

The overall reliability was .94, with reliabilities for the sub-scales ranging from .55 to .90. An alpha above 
.70 is generally considered acceptable, although .65 to .70 may be considered minimally acceptable peVellis, 
1991). Three sub-scales, Organizational Support for Customer Service, Dealing with Diverse Customers and 
Supporting Your Coworkers had alphas lower than .65. This is most likely due to the relatively small number of 
items in these subscales. 

Conclusion 

This study investigated the validity and reliability of an instrument to measure customer service competencies, and 
it provided cross-validation evidence of the SERVE™ competency model presented by Russ-Eft et al (2000). In 
doing so, it followed the level of-rigor standards for competency modeling and job analysis studies set forth by 
Shippmann et al (2000). 

Holton, Bates, Ruona, and Leimbach (1998) argue that such validation studies are critical for instruments 
used by HRD professionals. “With psychometrically strong instrumentation, HRD will be in a position to provide 
more definitive answers to questions. Without reliable instrumentation, researchers will be limited in their ability to 
arrive at conclusions and prescriptions” (p. 487 - 488). Although these authors were concerned about measuring 
transfer climate, the same arguments can be made for developing and validating instruments to measure job 
competencies. 
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Table 2. Cronbach’s alpha for each factor. 



Factor 


Alpha 


Number of Items 


Range of Item- 
Total 

Correlations 


Scale Mean (Std. 
Dev.) 


Providing Core Customer 
Service 


.90 


14 


.49-. 72 


6.24 (.55) 


Beyond Core Customer Service 


.81 


6 


.43-. 68 


5.60 (.89) 


Follow Through 


.78 


5 


.49-. 66 


5.63 (.98) 


Providing Information to 
Customers 


.75 


6 


.39-. 57 


6.07 (.63) 


Coping with Difficult 
Customers/ Situations 


.70 


4 


.45-. 52 


5.64 (.83) 


Being a Customer Service 
Advocate 


.74 


5 


.48-. 56 


5.54 (.96) 


Organizational Support for 
Customer Service 


.60 


4 


.27-. 46 


5.33 (.87) 


Dealing with Diverse 
Customers 


.58 


3 


.38-. 41 


5.86 (.92) 


Supporting Your Coworkers 


.55 


2 


.38 


6.15 (.81) 



The present study follows the recommendations of both Shippmann et all (2000) and Holton et al (2000) by 
supplementing the results of previous qualitative work using a critical incident method with a quantitative 
examination of the same concepts. 

Limitations 

All of the items in this instrument showed significant negative skew in the distribution of participant’s 
responses. While this did not seem to cause difficulties for the analyses presented here, it may continue to be an 
issue for the reporting of scale scores and other results. 

There are several possible explanations for the high responses to the items in this instrument. First, 622 
(73%) of respondents reported having received customer service training prior to completing the questionnaire. The 
high proportion of respondents receiving training could account, in part, for the high responses. A t-test was 
conducted to determine whether participants with prior training had higher scale scores. The difference between 
those with prior training and those without was small, but significant, (/ = 3.47, df = 814, p<.01). The mean scale 
score for those with prior training was 5.87, and 5.71 for those without prior training (on a seven-point scale). In 
one sense, this may present a limitation, in the sense that the data may have been affected by such difference. In 
another sense, this result may provide some additional evidence of validity in that it showed a significant difference 
between individuals who had had previous training and those who did not. 

Second, the skills identified as important to good customer service have a certain degree of social 
desirability. Social desirability is a problem when one response is more socially acceptable than another or when a 
respondent tends to report “good things” about himself or herself (Edwards, 1957). In this situation, a respondent 
may choose the socially desirable answer rather than the one that most closely reflects his or her behavior. This is 
not to suggest that people are lying regarding their behavior; rather, they believe that they are following socially 
accepted norms and answer accordingly. The problem of social desirability may be more likely to arise when a 
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questionnaire asks for respondents’ perspective on their own performance only. Social desirability may be less of an 
issue when a questionnaire reflects an organizational perspective or a second-party perspective (e.g., when 
managers rate their direct reports and not themselves). 

Re-framing the questions may help reduce the degree of social desirability in the scale. One option is to 
change the survey from the self-perspective to a group perspective. Such a change would ask participants to rate the 
practices of their customer service group as opposed to their own behavior. The respondents’ beliefs about their 
own skill levels might be tempered by their observations of their coworkers. Another option would be to include a 
social desirability rating for each service item. Customer service items that show strong correlations with the social 
desirability scale would then be considered for exclusion. However, if there is, indeed, a strong element of social 
desirability in the underlying construct itself, then exclusion of these items may undermine the quality of the entire 
instrument. 

Third, the study was undertaken with customer service personnel within the United States. We cannot be 
certain that the same results would emerge using personnel located in other countries and other cultures. This would 
require studies in separate countries and cultures. 

Implications for HRD Research and Practice 

Along with the work of Holton et al (1998), this study describes efforts to develop and validate standard instruments 
for use by HRD researchers and practitioners. Such studies can help other researchers as they examine 
competencies needed for specific roles or jobs within organizations, as well as other factors critical to the 
advancement of learning and performance within organizations. Furthermore, the present study provides an 
example of supplementing qualitative work in the area of competency analysis and assessment work with 
quantitative analyses. 

The instruments from such quantitative analyses examining reliabilities and factor structures provide 
practitioners with measurement tools to enhance learning and performance in organizations. Well-researched 
instruments can play a vital role in determining initial knowledge, skill, and competency gaps. Such information 
can assist practitioners to focus on the needs of individuals, groups, and organizations. 

In addition, such instruments provide critical tools in evaluating whether a specific learning and 
performance intervention resulted in targeted improvements. They can be used before and after training to 
determine competency improvements. They can also be used in post-training designs to ascertain whether 
competencies have reached a specified and pre-determined level. 

Finally, by using these instruments in both needs assessment and evaluation efforts, further research can be 
undertaken concerning the psychometric qualities of the instruments. So, with the customer service instruments, for 
example, data can also be gathered from customers through customer satisfaction surveys. Then one could 
undertake concurrent validation efforts comparing the results from the SERVE™ instruments with the customer 
satisfaction surveys. 

References 

Berry, L. L., Parasuraman, A., & Zeithaml, V. A. (1994). Improving service quality in America: Lessons 
learned. Academy of Management Executes (2), 32-52. 

Bhote, K. R. (1996). Bevond customer satisfaction to customer lovaltv: The key to greater profitability . 
New York: American Management Association. 

Bitner, M. J., Booms, B., &Tetreault, M. S. (1990, Jan.). The service encounter: Diagnosing favorable and 
unfavorable incidents. Journal of Marketing. 54 . 71-84. 

Corcoran, K.J., Petersen, L.K., Baitch, D.B., & Barrett, M.F. (1995L High performance sales organizations . 
Chicago: Irwin. 

DeVellis, R. F. (1991). Scale development; Theory and application . Newbury Park, CA: Sage. 

Edwards, A. L. (1957). The social desirability variable in personality assessment and research . New York: 

Holt. 

Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis (4 th ed.). 
Englewood Cliffs, NJ: Prentice Hall. 

Hart, C.W., & Johnson, M.D. (1999, Spring). Growing the trust relationship; Businesses begin raising the 
bar in search of 'total trust.' Marketing Management. 8 (1). 9. 



37-3 



24 



Holton, E. F. Ill, Bates, R. A., Ruona, W. E. A., & Leimbach, M. (1998). Development and validation of a 
generalized Learning Transfer Climate Questionnaire: Final report. In R. J. Torraco (Ed.), Academy of Human 
Resource Development 1998 Conference Proceedings. Baton Rouge, LA: AHRD. 

Jevons, C., & Pidgeon, J. (1999). Perceptions of customer opinions of service quality: A comparison 
between managers in Vietnam and Australia. International Management Development Association 8 th Annual World 
Business Congress Proceedings (pp. 177-183). Hummelstown, PA: International Management Development 
Association. 

Learning International (1991). Lessons from, top service providers: Turning a service vision into action . 
Stamford, CT: Learning International. 

Levesque, P. (1995). The Wow factor . Chicago: Irwin. 

Mandell, J. (1999, Dec. 2). It’s often hard to find help behind the cybercounter. The New York Times , p. 
G7. 

Nunnally, J. C., & Bernstein, 1. H. (1994). Psychometric theory . New York: McGraw-Hill. 

Russ-Eft, D., Berrey, C., Boone, T., & Winkle, J. (2000). When caring is not enough: Competencies needed 
by service providers. In IC P. Kuchinke (Ed.), Academy of Human Resource Development 2000 Conference 
Proceedings . Baton Rouge, LA: AHRD. 

Schippmann, J. S., Ash, R. A., Battista, M., Carr, L., Eyde, L. D., Hesketh, B., Kehoe, J., Pearlman, K., 
Prien, E. P., & Sanchez, J. I. (2000). The practice of competency modeling. Personnel Psychology. J3 . 703-740. 

Tabachnick, B. G., & Fidell, L. S. (1996). llsing^multi variate statistics . (3 rd ed.). New York, HarperCollins. 

Watemaux, C. M. (1976) Asymptotic distribution of the sample roots for a non-normal population. 
Biometrika. 63 (3L 639-645. 

Zemke, R. ( 1 995). W inning strategies: The 1 0 commandments of customer service. Incentive. 1 69 6 11), 81- 
83. 




37-3 

25 



Ce bSrt t'U- 

ACADEM Y OF HUMAN RESOURCE DEVELOPMENT 3 -7 _| 

2001 CONFERENCE PROCEEDINGS ^ — — 



Manuscript Release Form 



THIS FORM MUST BE 



COMPLETED AND RETURNED WITH EACH MANUSCRIPT. 



ONLY ONE AUTHOR IS REQUIRED TO SIGN THE FORM 



Paper Title 



Author Names 



An Examination 
(MIDAS) 



of the Multiple Intelligences Developmental Assessment Scales 



Albert K. Wiswell 
Christopher R. Hardy 
Thomas G. Reio, Jr. 



AHRD Reference * 


l uo — 

Please tell us where to communicate with you about this paper 


r'nntart nerson 


Chris R. Hardy, Ph.D. 


OUlIlQvt 

4- 

Address 


INNOLOG, Incorporated 
2010 Corporate Ridge 
5th Floor 

McLean VA 22102 
USA 


/■mn\ o^7 A 


Office Phone 
E-mail 


(703) 917-4404 
chardy@innolog.com 


Office Fax 


(703) 047-0/ ih 


We are adding a topical 
your paper. Examples it 
that will be ol the greate 


st help to your colleagues when they search for research on a topic. 


Key word 1 


multiple intelligences 


Key word 2 


assessment 


— 


Key word 3 


factor analysis 



breeding* writ) b* *ubr.Med .» ER.C bflbr .be eob.erence. The Ab.dem, require* your e.gne.ure to Include your 
paper in the Proceedings. 



prSy S ^?e'per S um. ^ daSesel?' *gn* gT S^wg or^^ "» - 




[1 



DATE 



C toil si* 

ACADEMY OF HUMAN RESOURCE DEVELOPMENT 
2001 CONFERENCE PROCEEDINGS 



Manuscript Release Form 



THIS FORM MUST BE COMPLETED AND RETURNED WITH EACH MANUSCRIPT . 
ONLY ONE AUTHOR IS REQUIRED TO SIGN THE FORM . 



Paper Title 


Evaluating Quality of Life Programs: Utilizing the Balanced Scorecard in Conjunction 
with Quasi-Experimental Design 


Author Names 


Wendy J. Casper 
Ann L. Landy 


AHRD Reference # 


096 




Please tell us where to communicate with you about this paper 



Contact person 


Ann Landy, Ph.D. 


Address 


Caliber Associates 
10530 Rosehaven Street 
Suite 400 

Fairfax VA 22030 ..... ... .... . _ . . . .. 

-USA- - - - .’ ... 


Office Phone 


(703)219-4324 


Office Fax 


(703) 385-3206 


E-mail 


landya@calib.com 



We are adding a topical index for the proceedings this year. Please list three key words that describe the primary topics 
of your paper. Examples might include teams, evaluation, diversity, performance measurement methods, etc. Choose 
words that will be of the greatest help to your colleagues when they search for research on a topic. 



Key word 1 


quality of life 


Key word 2 


evaluation 


Key word 3 


balanced scorecard 



The Proceedings will be submitted to ERIC after the conference. The Academy requires your signature to include your 
paper in the Proceedings. . 



I agree to allow Oscar A. Aliaga, editor of the 2001 Academy of Human Resource Development Proceedings, to submit the 
proceedings with my paper included to the ERIC database. By signing this I am releasing the paper for all authors of the paper. 




ERIC 



01/22/01 11:07 FAX 650 328 7518 



KINKO'S PALO ALTO 



c£ Oil fix 



7 5 : \£il 



ACADEMY OF HUMAN RESOURCE DEVELOPMENT 
2001 CONFERENCE PROCEEDINGS 



Manuscript Release Form 



THIS FORM MUST BE COMPLETED AND RETURNED WITH 
MANUSCRIPT. 

ONLY ONE AUTHOR IS REQUIRED TO SIGN THE FORM. 



Paper Title 



MERGEFIELD TITLE 

Validating a Tool to Measure Customer Service Competencies 
Author Names 



MERGEFIELD CONTACT 
Darlene Russ-Eft 
MERGEFIELD AUTHOR_2 
Caryl Berrey 

MERGEFIELD AUTHOR_3 
Regina Atwood 
MERGEFIELD AUTHOR_4 
Ruth Pangilinan 
AHRD Reference # 

MERGEFIELD PAPERID 

Please tell us where to communicate with you about this paper 



Contact person 

MERGEFIELD NAME 
Darlene Russ-Eft 
Address 

MERGEFIELD ORGANIZATION 
AchieveGlobal, Inc. 

MERGEFIELD DEPARTMENT 
Research Services 




@001 

77 



\/\) & 



01/22/01 11:07 FAX 650 328 7518 



KINKO'S PALO ALTO 



MERGEFIELD ADDRESS 
One Front Street, Suite 2050 
MERGEFIELD CITY 
San Francisco 
MERGEFIELD STATE 
California 

MERGEFIELD POSTAL 
94111 

MERGEFIELD COUNTRY 
U.S.A. 

Office Phone 



MERGEFIELD WORK 
(415)217-2687 
Office Fax 



MERGEFIELD FAX 
(415) 956-1496 
E-mail 

MERGEFIELD EMAIL 
Daxlene.russ-eft@acliievcglobal.com 



Key word 1 

MERGEFIELD KEYWORD 1 
customer service 
Key word 2 

MERGEFIELD KEYWORD2 
competencies assessment 
Key word 3 

MERGEFIELD KEYWORD3 
instrument validation 

The Proceedings will be submitted to ERIC after the conference. The Academy requires 
your signature to include your paper in the Proceedings. 



I agree to allow Oscar A. Aliaga, editor of the 2001 Academy of Human Resource 
Development Proceedings, to submit the proceedings with my paper included to the 
ERIC database. By signing this I am releasing the paper for all authors of the paper. 



/1>1 11: U# r AA 50U 5 2# 751# 



MI'RU 5 1-ALU ALU) 




Normal 
Normal 
Heading 1 
Heading 1 

Default Paragraph Font 

Default Paragraph Font 

Footnote Reference 

Footnote Reference 

Emphasis 

Emphasis 

Hyperlink 

Hyperlink 

Body Text 

Body Text 

-C:\Data\Ahrd\Conference 2001\AuthorNotice.xls 
-C:\Data\Ahrd\Conference 2001\AuthorNotice xls 

Carson R. Ametl:C:\Data\Ahrd\Conference 2001\Releases 2001\release_127.doc 
Carson R. Arnett: C:\Data\Ahrd\Confcrence 2001\Releases 2001\release 127.doc 
Carson R. Amett;C:\Data\Ahrd\Conference 200l\Relcases 2001\release3l28.doc 
Carson R. Amett:C:\Data\Ahrd\Conference 2001\Releases 2001\release_128.doc 
Carson R. Arnett:C:\Data\Ahrd\Conference 2001\Releases 2001\release_129.doc 
Carson R. Amett:C:\Data\Ahrd\Conference 2001\Releases 2001\release_129.doc 
Carson R. Arnett: C:\Data\Ahrd\Confercnce 200l\Releases 2001\release_130.doc 
Carson R. Arnett: C:\Data\Ahrd\Conference 2001\Releases 2001\release_130.doc 
Carson R. Arnett:C:\Data\Alird\Conference 2001\Releases 2001\release_I31.doc 
Carson R. Amett:C:VData\Ahrd\Conference 200l\Releases 2001\release_Ul.doc 
Carson R. Amett:C:\Data\Ahrd\Conference 2001AReleases 2001\release_132.doc 
Carson R. Amett: C:\Data\Ahrd\Conference 2001VReleases 2001\release 132.doc 
Carson R. Amett; C:\Data\Ahrd\Conference 2001\Releases 2001\release~133.doc 
Carson R. .Amett:C:\Data\Ahrd\Conferencc 200l\Releases 2001\release_133.doc 
Carson R. Amett:C:VDataVAhrd\Conference 2001\Releases 200l\release_134.doc 
Carson R. Arneit:C:\Data\Ahrd\Conference 2001\Releases 2001\release_134.doc 
Carson R. Amett:C:\Data\Ahrd\Conference 2001\Releascs 200l\release_135.doc 
Carson R. Amett. C:\Data\Ahrd\Conference 2001\Releases 2001\release_135.doc 



